Diffusion Model Research Articles

Mind-bridge: reconstructing visual images based on diffusion model from human brain activity

Mind-bridge: reconstructing visual images based on diffusion model from human brain activity

Diffusion models for conditional generation of hypothetical new families of superconductors

Effective computational search holds great potential for aiding the discovery of high-temperature superconductors (HTSs), especially given the lack of systematic methods for their discovery. Recent progress has been made in this area with machine learning, especially with deep generative models, which have been able to outperform traditional manual searches at predicting new superconductors within existing superconductor families but have yet to be able to generate completely new families of superconductors. We address this limitation by implementing conditioning—a method to control the generation process—for our generative model and develop SuperDiff, a denoising diffusion probabilistic model with iterative latent variable refinement conditioning for HTS discovery—the first deep generative model for superconductor discovery with conditioning on reference compounds. With SuperDiff, by being able to control the generation process, we were able to computationally generate completely new families of hypothetical superconductors for the very first time. Given that SuperDiff also has relatively fast training and inference times, it has the potential to be a very powerful tool for accelerating the discovery of new superconductors and enhancing our understanding of them.

Transcending classical diffusion models: nonlinear dynamics and solitary waves in the fractional Chaffee–Infante equation

Transcending classical diffusion models: nonlinear dynamics and solitary waves in the fractional Chaffee–Infante equation

Gloss-driven Conditional Diffusion Models for Sign Language Production

Sign Language Production (SLP) aims to convert text or audio sentences into sign language videos corresponding to their semantics, which is challenging due to the diversity and complexity of sign languages, and cross-modal semantic mapping issues. In this work, we propose a Gloss-driven Conditional Diffusion Model (GCDM) for SLP. The core of the GCDM is a diffusion model architecture, in which the sign gloss sequence is encoded by a Transformer-based encoder and input into the diffusion model as a semantic prior condition. In the process of sign pose generation, the textual semantic priors carried in the encoded gloss features are integrated into the embedded Gaussian noise via cross-attention. Subsequently, the model converts the fused features into sign language pose sequences through T-round denoising steps. During the training process, the model uses the ground-truth labels of sign poses as the starting point, generates Gaussian noise through T rounds of noise, and then performs T rounds of denoising to approximate the real sign language gestures. The entire process is constrained by the MAE loss function to ensure that the generated sign language gestures are as close as possible to the real labels. In the inference phase, the model directly randomly samples a set of Gaussian noise, generates multiple sign language gesture sequence hypotheses under the guidance of the gloss sequence, and outputs a high-confidence sign language gesture video by averaging multiple hypotheses. Experimental results on the Phoenix2014T dataset show that the proposed GCDM method achieves competitiveness in both quantitative performance and qualitative visualization.

Using Drift Diffusion and RL Models to Disentangle Effects of Depression On Decision-Making vs. Learning in the Probabilistic Reward Task

The Probabilistic Reward Task (PRT) is widely used to investigate the impact of Major Depressive Disorder (MDD) on reinforcement learning (RL), and recent studies have used it to provide insight into decision-making mechanisms affected by MDD. The current project used PRT data from unmedicated, treatment-seeking adults with MDD to extend these efforts by: (1) providing a more detailed analysis of standard PRT metrics—response bias and discriminability—to better understand how the task is performed; (2) analyzing the data with two computational models and providing psychometric analyses of both; and (3) determining whether response bias, discriminability, or model parameters predicted responses to treatment with placebo or the atypical antidepressant bupropion. Analysis of standard metrics replicated recent work by demonstrating a dependency between response bias and response time (RT), and by showing that reward totals in the PRT are governed by discriminability. Behavior was well-captured by the Hierarchical Drift Diffusion Model (HDDM), which models decision-making processes; the HDDM showed excellent internal consistency and acceptable retest reliability. A separate “belief” model reproduced the evolution of response bias over time better than the HDDM, but its psychometric properties were weaker. Finally, the predictive utility of the PRT was limited by small samples; nevertheless, depressed adults who responded to bupropion showed larger pre-treatment starting point biases in the HDDM than non-responders, indicating greater sensitivity to the PRT’s asymmetric reinforcement contingencies. Together, these findings enhance our understanding of reward and decision-making mechanisms that are implicated in MDD and probed by the PRT.

Modelling phytoplankton-virus interactions: phytoplankton blooms and lytic virus transmission

A dynamic reaction–diffusion model of four variables is proposed to describe the spread of lytic viruses among phytoplankton in a poorly mixed aquatic environment. The basic ecological reproductive index for phytoplankton invasion and the basic reproduction number for virus transmission are derived to characterize the phytoplankton growth and virus transmission dynamics. The theoretical and numerical results from the model show that the spread of lytic viruses effectively controls phytoplankton blooms. This validates the observations and experimental results of Emiliana huxleyi-lytic virus interactions. The studies also indicate that the lytic virus transmission cannot occur in a low-light or oligotrophic aquatic environment.

Fast Graph Generation via Spectral Diffusion.

Generating graph-structured data is a challenging problem, which requires learning the underlying distribution of graphs. Various models such as graph VAE, graph GANs, and graph diffusion models have been proposed to generate meaningful and reliable graphs, among which the diffusion models have achieved state-of-the-art performance. In this paper, we argue that running full-rank diffusion SDEs on the whole graph adjacency matrix space hinders diffusion models from learning graph topology generation, and hence significantly deteriorates the quality of generated graph data. To address this limitation, we propose an efficient yet effective Graph Spectral Diffusion Model (GSDM), which is driven by low-rank diffusion SDEs on the graph spectrum space. Our spectral diffusion model is further proven to enjoy a substantially stronger theoretical guarantee than standard diffusion models. Extensive experiments across various datasets demonstrate that our proposed GSDM turns out to be the SOTA model, by exhibiting both significantly higher generation quality and much less computational consumption than the baselines.

From hearing to seeing: Linking auditory and visual place perceptions with soundscape-to-image generative artificial intelligence

People experience the world through multiple senses simultaneously, contributing to our sense of place. Prior quantitative geography studies have mostly emphasized human visual perceptions, neglecting human auditory perceptions at place due to the challenges in characterizing the acoustic environment vividly. Also, few studies have synthesized the two-dimensional (auditory and visual) perceptions in understanding human sense of place. To bridge these gaps, we propose a Soundscape-to-Image Diffusion model, a generative Artificial Intelligence (AI) model supported by Large Language Models (LLMs), aiming to visualize soundscapes through the generation of street view images. By creating audio-image pairs, acoustic environments are first represented as high-dimensional semantic audio vectors. Our proposed Soundscape-to-Image Diffusion model, which contains a Low-Resolution Diffusion Model and a Super-Resolution Diffusion Model, can then translate those semantic audio vectors into visual representations of place effectively. We evaluated our proposed model by using both machine-based and human-centered approaches. We proved that the generated street view images align with our common perceptions, and accurately create several key street elements of the original soundscapes. It also demonstrates that soundscapes provide sufficient visual information places. This study stands at the forefront of the intersection between generative AI and human geography, demonstrating how human multi-sensory experiences can be linked. We aim to enrich geospatial data science and AI studies with human experiences. It has the potential to inform multiple domains such as human geography, environmental psychology, and urban design and planning, as well as advancing our knowledge of human-environment relationships.

FSRDiff: A fast diffusion-based super-resolution method using GAN

Single image super-resolution with diffusion probabilistic models (SRDiff) is a successful diffusion model for image super-resolution that produces high-quality images and is stable during training. However, due to the long sampling time, it is slower in the testing phase than other deep learning-based algorithms. Reducing the total number of diffusion steps can accelerate sampling, but it also causes the inverse diffusion process to deviate from the Gaussian distribution and exhibit a multimodal distribution, which violates the diffusion assumption and degrades the results. To overcome this limitation, we propose a fast SRDiff (FSRDiff) algorithm that integrates a generative adversarial network (GAN) with a diffusion model to speed up SRDiff. FSRDiff employs conditional GAN to approximate the multimodal distribution in the inverse diffusion process of the diffusion model, thus enhancing its sampling efficiency when reducing the total number of diffusion steps. The experimental results show that FSRDiff is nearly 20 times faster than SRDiff in reconstruction while maintaining comparable performance on the DIV2K test set.

Ion transport mechanism and diffusion model establishment in recycled aggregate concrete subjected to magnesium, sulfate, and chloride attack

Recycled aggregate concrete (RAC), which is used in Northwest China, is inevitably subjected to various aggressive environments. However, the aggressive ions of chloride, sulfate, and magnesium and the original ions of calcium and hydroxide in RAC under the combined action of magnesium-sulfate-chloride attack and dry–wet (D&W) cycles have received little attention in previous studies. Therefore, in this study, the content of aggressive and original ions in exposed RAC was quantified, and the influence of the replacement ratio of supplementary cementitious materials (SCMs) on diffusion was investigated. Results showed that the contents of chloride, sulfate, and magnesium in exposed RAC increased with an increase in the replacement ratio of silica fume (SF) and metakaolin (MK). In contrast, the contents decreased with the increase in the replacement ratio of granulated blast furnace slag (GBFS). At the same exposure time and depth, the content of RAC with ternary cementitious materials from the highest to the lowest was in the order of cement–fly ash –SF, cement–fly ash–MK, and cement–fly ash–GBFS. Visual appearance, relative compressive strength, and the diffusion coefficient exhibited a similar change law. An exponential relationship was observed between the diffusion and time delay coefficients and the replacement ratio of SCMs, and the content of aggressive ions on the exposed surface decreased linearly with increasing exposure time. Diffusion models of aggressive ions were established to consider the multiple effects. Aggressive ion–original ion interaction influenced the mineral phases’ composition in the corrosive zone and led to a difference in the diffusion process and depth.

Assessing brain microstructural changes in chronic kidney disease: a diffusion imaging study using multiple models

ObjectivesTo explore the effectiveness of diffusion quantitative parameters derived from advanced diffusion models in detecting brain microstructural changes in patients with chronic kidney disease (CKD).MethodsThe study comprised 44 CKD patients (eGFR<59 mL/min/1.73 m2) and 35 age-and sex-matched healthy controls. All patients underwent diffusion spectrum imaging (DSI) and conventional magnetic resonance imaging. Reconstructed to obtain diffusion MRI models, including diffusion tensor imaging (DTI), neurite orientation dispersion and density imaging (NODDI) and Mean Apparent Propagator (MAP)-MRI, were processed to obtain multi-parameter maps. The Tract-Based Spatial Statistics (TBSS) analysis was utilized for detecting microstructural differences and Pearson correlation analysis assessed the relationship between renal metabolism markers and diffusion parameters in the brain regions of CKD patients. Receiver operating characteristic (ROC) curve analysis assessed the diagnostic performance of diffusion models, with AUC comparisons made using DeLong’s method.ResultsSignificant differences were noted in DTI, NODDI, and MAP-MRI parameters between CKD patients and controls (p < 0.05). DTI indicated a decrease in Fractional Anisotropy(FA) and an increase in Mean and Radial Diffusivity (MD and RD) in CKD patients. NODDI indicated decreased Intracellular and increased Extracellular Volume Fractions (ICVF and ECVF). MAP-MRI identified extensive microstructural changes, with elevated Mean Squared Displacement (MSD) and Q-space Inverse Variance (QIV) values, and reduced Non-Gaussianity (NG), Axial Non-Gaussianity (NGAx), Radial Non-Gaussianity (NGRad), Return-to-Origin Probability (RTOP), Return-to-Axis Probability (RTAP), and Return-to-Plane Probability (RTPP). There was a moderate correlation between serum uric acid (SUA) and diffusion parameters in six brain regions (p < 0.05). ROC analysis showed the AUC values of DTI_FA ranged from 0.70 to 0.793. MAP_NGAx in the Retrolenticular part of the internal capsule R reported a high AUC value of 0.843 (p < 0.05), which was not significantly different from other diffusion parameters (p > 0.05).ConclusionThe advanced diffusion models (DTI, NODDI, and MAP-MRI) are promising for detecting brain microstructural changes in CKD patients, offering significant insights into CKD-affected brain areas.

Urban inundation mapping by coupling 1D − 2D models and model comparison

Urban inundation mapping is important for early warning, management, and drainage network planning. Urban flood simulations typically utilize a 1D model focused on pipe nodes, whereas 2D simulations are required over a land area. In this paper, our focus is on their coupling, so that a dynamic 1D-2D model is obtained. We couple the Storm Water Management Model (SWMM), a widely utilized urban drainage model, with 2D models to simulate urban inundation. The 1D-2D coupled inundation model uses the 1D model output, which represents the overflow volume of sparsely distributed pipe nodes. After coupling, surface inundation distribution maps are obtained. We developed a physically-based 2D overflow diffusion model (2D-DM) and compared it with two geostatistical models: Ordinary co-kriging (OCK) and Regression Kriging (RK). The study area is the central area of Haining City in China, where a severe storm occurred on August 3, 2018. The 2D-DM performed best (RMSE = 0.232), followed by RK (RMSE = 0.286), and OCK (RMSE = 0.361), but did not provide uncertainty measures, in contrast to the geostatistical models. We conclude that 1D-2D coupling was done successfully, with the different 2D models showing different qualities. To enhance urban inundation mapping, it is suggested to couple the 1D model with both physical and statistically based 2D models for a more comprehensive assessment of potential inundation areas and depths.

Attention-enhanced conditional-diffusion-based data synthesis for data augmentation in machine fault diagnosis

Data scarcity and class imbalance are pervasive challenges in machine fault diagnosis, impeding the development and broad adaptation of accurate and reliable deep-learning-based fault detection systems. To address these issues, we propose a novel attention-enhanced conditioning-guided diffusion-based approach for synthesizing additional training data. By generating noisy motor-current signals and employing a diffusion process to reduce noise levels, we obtain synthetic data that closely resembles the statistical properties and patterns of real-world data. The quality of the synthesized samples is assessed using a four-layer, pre-trained convolutional neural network, achieving an impressive classification rate of 96.39%. This demonstrates the effectiveness of our attention-enhanced conditioned diffusion-based model in synthesizing critical features for fault detection, making it well-suited for the introduced fault detection task. To further investigate the quality of the synthesized data samples, we implement a specific training strategy that uses the generated samples as training data. The trained diffusion model generates samples per class, which are then used to train a simple four-layer convolutional neural network for fault classification. Remarkably, the model achieves an accuracy of 98.96% when evaluated with real inverter signals, indicating that the generated samples effectively capture the distribution of real data. For further evaluation, multiple scenarios are created confirming the value of the additionally synthesized samples for real-world bearing fault detection. Our approach presents a well-suited solution to the data scarcity and class imbalance problem in machine fault diagnosis. The proposed method enhances the accuracy and reliability of fault diagnosis systems, offering valuable insights for various real-world condition monitoring applications.

A Synergistic Multi-Objective Evolutionary Algorithm with Diffusion Population Generation for Portfolio Problems

When constructing an investment portfolio, it is important to maximize returns while minimizing risks. This portfolio optimization can be considered as a multi-objective optimization problem that is solved by means of multi-objective evolutionary algorithms. The use of multi-objective evolutionary algorithms (MOEAs) provides an effective approach for dealing with the complex data involved in multi-objective optimization problems. However, current MOEAs often rely on a single strategy to obtain optimal solutions, leading to premature convergence and an insufficient population diversity. In this paper, a new MOEA called the Synergistic MOEA with Diffusion Population Generation (DPG-SMOEA) is proposed to address these limitations by integrating MOEAs with diffusion models. To train the diffusion model, a mixed memory pool strategy is optimized, which collects improved solutions from the MOEA/D-AEE, an optimized MOEA, as training samples. The trained model is then used to generate offspring. Considering the cold-start mechanism of the diffusion model, particularly during the training phase where it is not suitable for generating initial offspring, this paper adjusts and optimizes the collaborative strategy to enhance the synergy between the diffusion model and MOEA/D-AEE. Experimental validation of the DPG-SMOEA demonstrates the advantages of using diffusion models in low-dimensional and relatively continuous data analysis. The results show that the DPG-SMOEA performs well on the low-dimensional Hang Seng Index test dataset, while achieving average performance on other high-dimensional datasets, consistent with theoretical predictions. Overall, the DPG-SMOEA achieves better results compared to MOEA/D-AEE and other multi-objective optimization algorithms.

StructDiffusion: End-to-end intelligent shear wall structure layout generation and analysis using diffusion model

Shear wall structures are widely used in high-rise buildings. However, the design process of shear wall structures suffers from inefficiencies. This paper introduces StructDiffusion, an end-to-end intelligent system for shear wall layout generation and analysis. The proposed approach tackles the layout design task by employing a novel diffusion model architecture to generate conditional images. Key components of the proposed method include pretrained diffusion models, ControlNet for conditional control, and LoRA for efficient adaptation. Notably, the method allows for architectural plan images and basic textual design conditions as inputs, enabling manipulation of the generated layouts by adjusting attributes such as building height and seismic intensity. To evaluate the quality of the model's design swiftly, this paper presents a comprehensive evaluation framework that incorporates perceptual and structural validity metrics. Through experimental analyses, this paper demonstrates the effectiveness of the proposed model in generating layouts, surpassing the capabilities of GAN-based methods. Furthermore, our study investigates model-specific parameter configuration, text-guided capabilities, code compliance, and computational efficiency. StructDiffusion significantly enhances the automation and intelligence of structural engineering workflows. The framework effectively addresses challenges associated with instability, data scarcity, and limitations in assessment. This pioneering application of diffusion models opens up promising avenues for advancing data-driven structural design.

State of the Art on Diffusion Models for Visual Computing

AbstractThe field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes. In these domains, diffusion models are the generative AI architecture of choice. Within the last year alone, the literature on diffusion‐based tools and applications has seen exponential growth and relevant papers are published across the computer graphics, computer vision, and AI communities with new works appearing daily on arXiv. This rapid growth of the field makes it difficult to keep up with all recent developments. The goal of this state‐of‐the‐art report (STAR) is to introduce the basic mathematical concepts of diffusion models, implementation details and design choices of the popular Stable Diffusion model, as well as overview important aspects of these generative AI tools, including personalization, conditioning, inversion, among others. Moreover, we give a comprehensive overview of the rapidly growing literature on diffusion‐based generation and editing, categorized by the type of generated medium, including 2D images, videos, 3D objects, locomotion, and 4D scenes. Finally, we discuss available datasets, metrics, open challenges, and social implications. This STAR provides an intuitive starting point to explore this exciting topic for researchers, artists, and practitioners alike.

Facile synthesis of amine hybrid silica aerogel globule via wet casting assisted self-catalysed sol-gel process for CO2 capture including direct air capture

Amine hybrid silica aerogel globule (AHSAG) for CO2 capture including direct air capture (DAC) was firstly synthesized via wet casting assisted self-catalyzed sol-gel process. AHSAG has abundant interpenetrating macropores, which favors diffusion of CO2 into internal amine sites and can enhance CO2 adsorption kinetics and capacity. The optimal CO2 adsorption capacity of AHSAG with dry 100% and 400 ppm CO2 are 3.82 (30 °C) and 1.86 mmol/g (55 °C), respectively. Corresponding amine efficiencies are 0.478 and 0.233. The effect of temperature on CO2 adsorption capacity is attributed to simultaneous enhancement of mass transfer kinetics and adsorption thermodynamics. The highest CO2 adsorption rate of AHSAG is achieved at 70 and 55 °C with 100% and 400 ppm CO2, respectively. The trend of CO2 adsorption rate with temperature is attributed to the synergistic effect of diffusion kinetics and desorption. Based on experimental CO2 adsorption kinetics, adsorption mechanism in 400 ppm CO2 was estimated with different diffusion models. Both film diffusion resistance and intraparticle diffusion resistance are involved in CO2 capture of AHSAG from air under given condition to control the overall adsorption process. Three kinetics models were used to fit the experimental data of AHSAG for DAC, and the Avrami model showed the best fitting within the whole adsorption period. Comparing with its state-of-the-art counterparts, AHSAG offers significant advantages for practical DAC, such as high adsorption capacity and amine efficiency, fast adsorption kinetics, and excellent cyclic stability.

Beta Brownian Motion

Abstract Brownian Motion (BM) is the paradigmatic model of diffusion. Transcending from diffusion to anomalous diffusion, the prominent Gaussian generalizations of BM are Scaled BM (SBM) and Fractional BM (FBM). In the sub/super diffusivity regimes: SBM is characterized by aging/anti-aging, and FBM is characterized by anti-persistence/persistence. BM is neither aging/anti-aging, nor persistent/anti-persistent. Within the realm of diffusion, a recent Gaussian generalization of BM, Weird BM (WBM), was shown to display aging/anti-aging and persistence /anti-persistence. This paper introduces and explores the anomalous-diffusion counterpart of WBM, Beta BM (BBM), and shows that: the weird behaviors of WBM become even weirder when elevating to BBM. Indeed, BBM displays a rich assortment of anomalous behaviors, and an even richer assortment of combinations of anomalous behaviors. In particular, the BBM anomalous behaviors include aging/anti-aging and persistence/anti-persistence -- which BBM displays in both the sub and super diffusivity regimes. So, anomalous behaviors that are unattainable by the prominent models of SBM and FBM are well attainable by the BBM model.

Accelerated motion correction with deep generative diffusion models.

The aim of this work is to develop a method to solve the ill-posed inverse problem of accelerated image reconstruction while correcting forward model imperfections in the context of subject motion during MRI examinations. The proposed solution uses a Bayesian framework based on deep generative diffusion models to jointly estimate a motion-free image and rigid motion estimates from subsampled and motion-corrupt two-dimensional (2D) k-space data. We demonstrate the ability to reconstruct motion-free images from accelerated two-dimensional (2D) Cartesian and non-Cartesian scans without any external reference signal. We show that our method improves over existing correction techniques on both simulated and prospectively accelerated data. We propose a flexible framework for retrospective motion correction of accelerated MRI based on deep generative diffusion models, with potential application to other forward model corruptions.

A toxicant-taxis reaction–diffusion model for the interaction between two species and a toxicant in an aquatic ecosystem

In this paper, we develop a toxicant-taxis reaction–diffusion model that describes the interaction dynamics between two species and a toxicant in a polluted aquatic ecosystem. We analyze the existence and stability of the equilibria of the corresponding space-absent ordinary differential equation system. We then perform linear stability analysis for our reaction–diffusion system to obtain sufficient conditions on model parameters under which the stable spatially homogeneous steady states become unstable. Numerical simulations are performed to illustrate that spatially inhomogeneous patterns and spatiotemporal patterns between the two species and the toxicant will typically emerge. We also numerically analyze the effects of spatially varying toxicant input and the intensity of toxicant-taxis on the population persistence and species diversity. It turns out that heterogeneous toxicant inputs and substantial toxicant taxis can boost the coexistence of two species, thereby promoting species diversity in ecosystems. By comparing the impact of a toxicant on a single species versus its impact on two species, we find that the collective sharing of the toxic burden among multiple species proves advantageous for biodiversity.